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DETAILED ACTION 

1 . The text of those sections of Title 35, U.S. Code not included in this action can be found 
in a prior Office action. 

Response to Amendment 

2. This communication is responsive to the applicant's amendment dated 02/07/2008. The 
applicant(s) amended claims 1 and 9 (see the amendment: pages 2-4). 

The examiner withdraws the previous claim rejection under 35 USC 1 12 2 nd , because the 
applicant amended the corresponding claim(s). 

Response to Arguments 

3. Applicant's arguments filed on 02/07/2008 with respect to the claim rejection under 35 
USC 103, have been fully considered but are moot in view of the new ground(s) of rejection, 
since the amended claims introduce new issue/matter and change the scope of the claims. It is 
noted that the combined prior art still applicable to the amended claims under new ground 
rejection (see below). It is also noted that the applicant's arguments regarding the unamended 
limitations/claims have been fully considered but they are not persuasive (see below). 

4. In response to applicant's arguments with respect to claim 1 (also applied to claims 9 and 
17) that "there is nothing in the teachings of column 4 (Attwater) that would lead one of skill in 
the art to . . .believe or equate the predefined set of meaning associated the operation of the 
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classifier or spoken language understanding to refer to "call types" as that term is used in claim 
1", "this concept of a call type is not found within the predefined set of meanings of Attwater et 
al." , "the process which is described in column 6 certainly does not involve modifying a 
plurality of call types. . ." (Remarks: page 8, paragraph 1 to page 10, paragraph 2), the examiner 
respectfully disagrees with applicant's arguments and has a different view of prior art teachings 
and claim interpretations. 

Firstly, it is noted that the applicant's arguments only attract one cited reference, and 
totally silenced the teachings regarding the clustered/classified "call type" disclosed by the other 
reference (ARAI). In response to applicant's arguments against the references individually, one 
cannot show nonobviousness by attacking references individually where the rejections are based 
on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In 
re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). 

Secondly, it is noted that ATTWATER discloses 'recognizer . . . provides as an output a 
graph which represents sequences of words or sub-words which most closely resemble the 
received speech utterance', 'the output graph.. .are received by a classifier, which classifies the 
received graph according to according to a predefined set of meanings, with reference to a 
semantic model to form a semantic classification (col. 3, line 60 to col. 4, line 14), so that one of 
ordinary skill in the art would have recognized that when the received speech utterances involve 
call types the classifier would classify the graph associated with a call type according to a 
predefined set of meanings. Indeed, ATTWATER does disclose 'measuring the occurrence of 
particular types (call types) of telephone enquiry received in a call centre' (col. 3, lines 4-6), 
using 'data read from a corpus of call example' (col. 5, lines 35-37), 'determine the number and 
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types of calls', which clearly support or suggest that the processed speech directly/indirectly 
involves or highly relates to determining "call type", and provide strong motivation for combine 
the references. Similarly, since ATTWATER teaches that 'the sentences in supervised training 
corpus 42 are clustered using clustering algorithm' and 'clusters thus generated are manually 
checked' in which the words/phrases can be deleted or substituted (modified) in forming a 
cluster (col. 6, lines 1-22), one of ordinary skill in the art would have recognized that supervised 
training with manually checked clusters would provide capability of modifying the 
clusters/classes for the transcribed data so as to form a different a language model (i.e. second 
NLU model). 

Thirdly, it also noted that the test for obviousness is not whether the features of a 
secondary reference may be bodily incorporated into the structure of the primary reference; nor 
is it that the claimed invention must be expressly suggested in any one or all of the references. 
Rather, the test is what the combined teachings of the references would have suggested to those 
of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). In this 
case, both reference teaches using speech recognition, language model, clustering/classifying 
speech utterances including semantic/meaning association with call types (as stated above). 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention 
was made to modify ARAI by combining the feature of using a candidate set of grammar 
fragments associating the clustered call-types in the training transcription disclosed by ARAI 
(col. 3, lines 1-60) and the feature of using supervised training and/or manually checking (or 
transcribing) clusters with modifying capabilities, such as deleting or substituting, as taught by 
ATTWATER (col. 6, lines 1-22), so that the call type of the candidate fragments associating the 
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utterances/transcription can be manually modified and another fully expanded salient fragment 
network (second natural language understanding model) can be generated, for the purpose 
(motivation) of generating more accurate transcriptions and/or improving call-type classification 
performance for the system (ATTWATER: col. 6, lines 30-31; ARAL col. 10, lines 21-22). 

Regarding rest of dependent claims, the response to the applicant argument (Remarks: 
page 10, paragraph 2-5) is based on the same reason stated above, because the arguments are 
based on the same issue(s) as their independent claims (see above). 

For above reasons, the rejection is sustained. 

Claim Rejections - 35 USC § 112 
The following is a quotation of the first paragraph of 35 U.S.C. 1 12: 

The specification shall contain a written description of the invention, and of the manner and process of making 
and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode 
contemplated by the inventor of carrying out his invention. 

5. Claims 9-16 are rejected under 35 U.S.C. 1 12, first paragraph, as failing to comply with 
the written description requirement. The claim(s) contains subject matter which was not 
described in the specification in such a way as to reasonably convey to one skilled in the relevant 
art that the inventor(s), at the time the application was filed, had possession of the claimed 
invention. 

Regarding claim 9, the limitations of "a natural language understanding model that 
receive(s) input from the automatic speech (recognition) module" and "a spoken language 
synthesizing module that receives input from the natural language module to synthesize a 
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spoken response" introduce new subject matter, which is not specifically described/disclosed in 
the original specification. 

Regarding claims 10-16, the rejection is based on the same reason described for claim 9, 
because the dependent claims include the same or similar problematic limitation(s) as claim 9. 

Claim Rejections - 35 USC § 103 

6. Claims 1-3, 5-6, 8-11, 13-14 and 16-22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over ARAI (US 6,173,261) in view of ATTWATER et al. (US 6,839,671) 
hereinafter referenced as ATTWATER. 

As per claim 1, ARAI discloses 'grammar fragment acquisition using syntactic and 
semantic clustering' (title) 'for recognizing and understanding fluently spoken languages' 
(abstract), comprising: 

"collecting a plurality of utterances" (Fig. 9 and col. 9, lines 14-8, 'database (collection) 
of a large number of utterances'); 

"generating a plurality of call types, each generated call type being based on a first set of 
utterances selected from said collection of utterances, (col. 2, lines 13-35, 'clustering phrases 
into grammar fragments' that are associated to the utterances, 'generate a collection of grammar 
fragments each representing a set of syntactically and semantically similar phrases' and used to 
'determine a call classification (a call type)'; Fig. 9 and col. 9, line 1 to col. 10, line 45, 'a set of 
candidate phrases having a probabilistic relationship with one or more of the set of 
predetermined routing objectives (including call types) with which the input speech utterances 
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are associated', 'call-type classification (generating call types)'; Fig. 2, also showing call types 
having/associating the training transcriptions (corresponding to the first set of utterances)); 

"generating a first natural language understanding model using call type information 
contained within said first set of utterances"(col. 2, lines 6-9 and 20-35, 'to utilize these grammar 
fragments (associating corresponding utterances) in language models (interpreted as natural 
language understanding models) for both speech recognition and understanding', 'salient 
sequences of these fragments may then be automatically acquired, which are then exploited by a 
spoken understanding module to determine a call classification'; Figs. 1 1 a- 1 1 c and col.10, lines 
30-45, 'as a consequence of this expansion, a fully expanded salient fragment network (also 
corresponding to the first natural language understanding model) is obtained (generated)'); 

"testing said first natural language understanding model" (col. 9, lines 61-67, 'recognition 
language model (natural language understanding model)', 'the training transcription contained 
7,800 sentences while the test transcription contained 1000 sentences', which implies testing the 
language model); 

Even though ARAI discloses that the grammar fragments formed from candidates 
phrases that generated from the training transcription (based on the testing) can be sorted based 
on call types (col. 6, lines 39-53), ARAI does not expressly disclose "modifying said plurality of 
call types based on said testing" and "generating a second natural language understanding 
model using said modified plurality of call types". However, the feature is well known in the art 
as evidenced by ATT WATER who discloses Teaming of dialogue states and language model of 
spoken information system' (title) for creating 'a dialog model' using a training corpus of 
example human-human dialogues (abstract), comprising 'a natural language call steering system' 
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in that 'the received speech utterance is analysed by the recognizer with reference to a language 
model' and using 'semantic model to form a semantic classification' that provides classifiers 
according to a predefined set of meanings (corresponding to call types) (col. 3, line 60 to col. 4, 
line 14), and teaches that 'the sentences in supervised training corpus 42 are clustered using 
clustering algorithm' and 'clusters thus generated are manually checked' in which the 
words/phrases can be deleted or substituted (modified) in forming a cluster (col. 6, lines 1-22). 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention 
was made to recognize that supervised training with manually checked clusters would provide 
capability of modifying the clusters/classes for the transcribed data so as to form a different 
language model, and to modify ARAI by combining the feature of using a candidate set of 
grammar fragments associating the clustered call-types in the training transcription disclosed by 
ARAI (col. 3, lines 1-60) and the feature of using supervised training and/or manually checking 
(or transcribing) clusters with modifying capabilities, such as deleting or substituting, as taught 
by ATT WATER (col. 6, lines 1-22), so that the call type of the candidate fragments associating 
the utterances/transcription can be manually modified and another fully expanded salient 
fragment network (second natural language understanding model) can be generated, for the 
purpose (motivation) of generating more accurate transcriptions and/or improving call-type 
classification performance for the system (ATTWATER: col. 6, lines 30-31; ARAI: col. 10, lines 
21-22). 

As per claim 2 (depending on claim 1), ARAI in view of ATTWATER further discloses 
"generating an annotation guide using a second set of utterances which is a subset of said first set 
of utterances" (ATTWATER: Fig. 3 and col. 5, lines 13-14, 'nodes... have been annotated with 
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operator utterance'; col. 11, lines 33-35, 'each call in the corpus can be annotated according to 
the cluster of each operator utterance in the call', wherein the content of labels 26 in Fig.3, such 
as 'greeting', can be reasonably interpreted as generated annotation guide as claimed; ARAL 
Fig.l 1C also suggests that the utterances corresponding to phrase 'collect call' (or 'collect phone 
call') is a subset of the utterances of the consequence expansion (the model), so that it would 
have been obvious to one of ordinary skill in the art at the time the invention was made to 
combine the teachings from ARAI and ATTWATER by providing generated annotation 
(annotation guide) using a subset of utterances of the consequence expansion, for the purpose 
(motivation) of generating more accurate transcriptions and/or improving call-type classification 
performance for the system (ATTWATER: col. 6, lines 30-31; ARAI: col. 10, lines 21-22)). 

As per claim 3 (depending on claim 1), ARAI in view of ATTWATER further discloses 
"generating call type data using at least one of data clustering, relevance feedback, string 
searching, data mining, and active learning tools" (ARAI: Fig. 9, 'grammar fragment (data) 
clustering'; ATTWATER: col. 5, lines 61-65, 'dynamic programming (DP) match (string 
searching)'). 

As per claim 5 (depending on claim 1), ARAI in view of ATTWATER further discloses 
"said first natural language understanding model is trained using a first text file containing 
utterances contained within said first set of utterances and a second text file containing call types 
assigned to said utterances in said first text file" (ARAI: Fig. 9 and col. 9, line 4 to col. 10, line 
45, wherein the 'database' with labeled utterances and training transcriptions necessarily include 
text file/table (first text file) linking (containing) the corresponding utterances, and the phases 
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(text) classified with call types are also necessarily stored in a file or table (second text file) and 
linked (assigned) to the corresponding utterances; also see Figs. 7A-7C and 1 1A-1 1C). 

As per claim 6 (depending on claim 1), ARAI in view of ATTWATER further discloses 
"said natural language understanding model is tested using a subset of said first set of utterances" 
(ARAI: Fig. 9, 'test speech utterance' and 'input speech'). 

As per claim 8 (depending on claim 1), ARAI in view of ATTWATER further discloses 
"said first natural language understanding model is created prior to an annotation guide" 
(ATTWATER: Fig. 3, wherein the content of labels 26, such as 'greeting', is interpreted as 
generated annotation guide; col. 1 1, lines 33-35, 'once the sentences in the training database 
have been clustered . . .each call in the corpus can be annotated according to the cluster of each 
operator utterance in the call', which suggests the model is created prior to the annotation 
(guide)). 

As per claims 9-11, 13-14 and 16, as best understood in view of the claim rejection 
under 35 USC 112 1st (see above), the rejection is based on the same reason described for claims 
1-3,5-6 and 8, because the claims recite the same or similar limitation(s) as claims 1-3,5-6 and 8 
respectively, wherein the combined teachings of a mechanism for 'speech recognition' (or 'the 
engine used for speech recognition') and 'speech synthesiser' (ARAI: col. 2, lines 6; col. 9, lines 
61); ATTWATER: col. 3, line 60 to col. 4, line 23) are read on the claimed "automatic speech 
recognition module" and "spoken language synthesizing module". 

As per claims 17-20, the rejection is based on the same reason described for claims 1-2 
and 5-6, because it also reads on the limitations of claims 1-2 and 5-6 respectively. 
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7. Claims 4, 7, 12 and 15 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
APvAI in view of ATTWATER as applied to claims 1 and 9, and further in view of MAES et al. 
(US 2003/0088421) hereinafter referenced as MAES. 

As per claim 4 (depending on claim 3), even though ARAI in view of ATTWATER 
discloses generating call types, as stated above, ARAI in view of ATTWATER does not 
expressly disclose "using a graphical user interface (GUI)." However, the feature is well known 
in the art as evidenced by MAES who discloses 'application that supports multi-modal', 
'conversational applications' utilizing 'NLU (natural language understanding)', 'multi -modal 
interactive dialog comprises modalities such as speech, visual (GUI)... and a combination of such 
modalities (e.g. speech and GUI)' (p(paragraph)46); and 'multi -modal browser application 
comprise a GUI browser' (p73). Therefore, it would have been obvious to one of ordinary skill 
in the art at the time the invention was made to modify ARAI in view of ATTWATER by 
combining the feature of generating call types as stated for claims 1 and 3, with feature of 
supporting multi-modal applications including using GUI, as taught by MAES, for the purpose 
(motivation) of better disambiguating and understanding the user's intention and/or displaying 
the related presenting and updating information (MAES: p46, p244). 

As per claim 7 (depending on claim 1), the rejection is based on the same reason 
described for claim 4, because the claim recites the same or similar limitation(s) as claim 4. 

As per claim 12 (depending on claim 1 1), the rejection is based on the same reason 
described for claim 4, because the claim recites the same or similar limitation(s) as claim 4. 

As per claim 15 (depending on claim 9), the rejection is based on the same reason 
described for claim 4, because the claim recites the same or similar limitation(s) as claim 4. 
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Conclusion 

8. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A 
shortened statutory period for reply to this final action is set to expire THREE MONTHS from 
the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the 
mailing date of this final action and the advisory action is not mailed until after the end of the 
THREE-MONTH shortened statutory period, then the shortened statutory period will expire on 
the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will the statutory 
period for reply expire later than SIX MONTHS from the date of this final action. 



9. Please address mail to be delivered by the United States Postal Service (USPS) as 
follows: 

Mail Stop 

Commissioner for Patents 

P.O. Box 1450 

Alexandria, VA 22313-1450 
or faxed to: 571-273-8300, (for formal communications intended for entry) 
Or: 571-273-8300, (for informal or draft communications, and please label 
"PROPOSED" or "DRAFT") 

If no Mail Stop is indicated below, the line beginning Mail Stop should be omitted from 



Effective January 14, 2005, except correspondence for Maintenance Fee payments, 
Deposit Account Replenishments (see 1.25(c)(4)), and Licensing and Review (see 37 CFR 5.1(c) 
and 5.2(c)), please address correspondence to be delivered by other delivery services (Federal 
Express (Fed Ex), UPS, DHL, Laser, Action, Purolater, etc.) as follows: 

U.S. Patent and Trademark Office 

Customer Window, Mail Stop 

Randolph Building 
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Alexandria, VA 223 14 
Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Qi Han whose telephone numbers is (571) 272-7604. The 
examiner can normally be reached on Monday through Thursday from 9:00 a.m. to 7:30 p.m. If 
attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Richemond Dorvil, can be reached on (571) 272-7602. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Inquiries regarding the status of submissions 
relating to an application or questions on the Private PAIR system should be directed to the 
Electronic Business Center (EBC) at 866-217-9197 (toll-free) or 703-305-3028 between the 
hours of 6 a.m. and midnight Monday through Friday EST, or by e-mail at: ebc@uspto.gov. For 
general information about the PAIR system, see http://pair-direct.uspto.gov. 

QH/qh 

April 17, 2008 
/Richemond Dorvil/ 

Supervisory Patent Examiner, Art Unit 2626 



